Stamp delta bundles with base commit SHA and exclude stale artifact DBs#27
Open
joshfriend wants to merge 1 commit into
Open
Stamp delta bundles with base commit SHA and exclude stale artifact DBs#27joshfriend wants to merge 1 commit into
joshfriend wants to merge 1 commit into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f71dfb01af
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
joshfriend
force-pushed
the
jfriend/delta-base-commit-stamping
branch
3 times, most recently
from
0dcf412 to
139f811
Compare
joshfriend
changed the title
joshfriend
force-pushed
the
jfriend/delta-base-commit-stamping
branch
from
139f811 to
91ed146
Compare
joshfriend
changed the title
joshfriend
force-pushed
the
jfriend/delta-base-commit-stamping
branch
8 times, most recently
from
8ce44ad to
df1c98a
Compare
Embed the base commit SHA in delta bundles to prevent cross-base contamination. On restore, write the hit commit to .cache-base-commit. SaveDelta reads it and stamps the delta via S3 metadata and a synthetic __base_commit__ tar entry. Restore checks both before applying. Also exclude module-artifact.bin and resource-at-url.bin from deltas to prevent stale resolution metadata from causing missing-jar failures.
joshfriend
force-pushed
the
jfriend/delta-base-commit-stamping
branch
from
df1c98a to
fe0d968
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Delta base-commit stamping
Prevents cross-base contamination by embedding the base commit SHA in delta bundles.
Save path:
SaveDeltareads.cache-base-commitfrom GRADLE_USER_HOME and stamps the delta with the base SHA in two places:x-amz-meta-base-commit) for cheap pre-download check via HEAD__base_commit__tar entry as first archive entry (fallback for non-S3 backends)Restore path:
RestorewriteshitCommitto.cache-base-commitafter base extraction. The delta pre-fetch goroutine checks metadata before downloading and tar stamp after downloading. Mismatches skip the delta entirely.Standalone
RestoreDelta: checks metadata against the local.cache-base-commitfile.Extraction:
processEntryskips the synthetic__base_commit__entry so it never hits the filesystem.Delta exclusions
Adds
module-artifact.binandresource-at-url.bintoDeltaExclusions. Both are Gradle embedded BTree databases rewritten every build due to compaction. A stalemodule-artifact.binfrom one resolution applied on top of a base from a different resolution causes Gradle to skip downloading jars it thinks are cached, then fail withFile/directory does not exist.Integration test
Adds
TestIntegrationDeltaConfigurationCacheStaleArtifactPathreproducing the advancing-base scenario where a delta from Base A applied to Base C causes stale configuration-cache paths.